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ABSTRACT 


During  the  period  from  October  1978  to  March  1979,  £he  CRC  sponsored  a 
test  program  to  compare  the  performance  of  aircraft  emission  measure¬ 
ment  systems.  The  tests  were  conducted  at  the  FAA's  National  Aviation 
Facilities  Experimental  Center^NAFECh^Atlantic  City,  Nevr-dersey. 
This  report  details  the  results  of  an  analysis  of  the  secured  cross¬ 
correlation  emissions  data  taken  over <£-pe*4ed-e^ five  weeks  by  seven 
participants.  Results^MK^at^-tirat^the  parti ci pant-to-parti'ci pant 
variability  of  the  gaseous  pol  lutants  <weee^  < ,  ■  j'i  *  i  dsa  -  .  — - 
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^Results  for  smoke  indicate  additional  development  effort  is  required 
to  secure  repeatable  smoke  data  among  participants.  The  smoke 
analysis  was  also  limited  by  the  small  amount  of  data  and  the  high 
levels  which  are  no  longer  typical  of  current  engine  emission  control 
technology. 
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I.  INTRODUCTION  AND  BACKGROUND 


In  order  to  meet  a  growing  need  to  determine  the  degree  of  correlation 
between  aircraft  gas  turbine  emission  and  smoke  measurements  made  by 
various  test  facilities,  the  Coordinating  Research  Council,  Inc.  (CRC) 
made  a  request  to  the  Federal  Aviation  Administration  (FAA)  to  conduct 
a  cooperative  emissions  testing  program.  The  FAA  accepted  the 
responsibility  and  agreed  to  host  the  testing  program  at  the  National 
Aviation  Facilities  Experimental  Center  (NAFEC),  now  the  FAA  Technical 
Center,  near  Atlantic  City,  New  Jersey. 

Several  organizations  were  invited  to  participate,  but  it  was  found 
that  some  of  them  were  using  stationary  instrumentation  systems  which 
could  not  be  moved  to  Atlantic  City.  The  program  was  then  divided 
into  two  phases.  Phase  I  was  to  be  conducted  at  Atlantic  City  for 
mobile  or  transportable  emissions  measurements  systems.  Phase  II 
would  correlate  results  from  stationary  or  non- transportable  systems, 
and  would  be  conducted  at  the  facilities  of  the  four  participants  in 
this  phase.  The  FAA  would  send  one  of  their  mobile  emissions  research 
facilities  (MERF)  to  each  of  the  locations  to  provide  the  reference 
measurements  in  each  case.  The  Phase  I  work  took  place  between 
October  1978  and  March  1979.  The  Phase  II  work  was  done  between  June 
and  August  1979. 

A  third  phase  was  conducted  under  CRC  scrutiny  in  the  United  Kingdom. 
The  National  Gas  Turbine  Establishment  (now  the  Royal  Aircraft 
Establishment)  and  Rolls-Royce  carried  out  a  two-week  program  at 
Derby,  England,  in  May  1979. 

This  report  will  discuss  only  the  work  conducted  under  Phase  I.  The 
results  of  Phase  II,  and  of  the  British  program  are  the  basis  of 
separate  reports. 

The  CRC  provided  a  Test  Coordinator,  Mr.  E.  F.  Marshall,  to  handle  the 
scheduling  and  data  collection,  and  to  oversee  the  subsequent  analysis 
of  results.  The  CRC  Test  Coordinator  was  present  during  the  Phase  I 
and  Phase  II  testing  and  also  during  the  British  testing  program. 

The  data  from  Phase  I  were  entered  into  a  computer  program  by  the  FAA 
and  several  standard  statistical  calculations  were  performed.  The 
results  of  this  work(1,2)  are  voluminous  and  are  available  for  inspec¬ 
tion  at  the  CRC  office.  In  this  report,  the  results  are  analyzed  and 
reported  in  accordance  with  the  specific  test  program  objectives. 

Membership  of  the  CRC  Aviation  Study  Group  on  Aircraft  Exhaust,  which 
was  responsible  for  overseeing  the  program,  is  listed  in  Appendix  A. 
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II.  TEST  OBJECTIVES 


The  major  objectives  of  the  FAA  test  program  were  as  follows: 


1.  To  establish  a  cross-correlation  of  government  and  engine  manu¬ 
facturers'  smoke  and  gaseous  emissions  measurement  systems 
operating  in  a  normal  test  configuration. 


2.  To  determine  the  typical  variability  observed  between  testing 
groups,  or  participants,  in  a  normal  test  configuration  while 
sampling  on  an  actual  turbine  engine. 


3.  To  estimate  the  precision  errors  for  the  participants  and  to 
evaluate  the  effect  of  that  precision  error  on  the  observed 
variability. 


4.  To  estimate  the  bias  introduced  into  emissions  measurements  by 
span  gases  and  to  ensure  that  corrections  for  span  gases  are 
available. 


In  this  report,  these  objectives  are  addressed  for  the  Phase  I  testing 
at  NAFEC  only.  The  variability  reported  is  carefully  restricted  to 
the  test  objective  cited.  Little  attention  is  paid  to  assessing 
reasons  for  the  observed  variability. 


III.  PARTICIPANTS 


Table  I  lists  the  participants  in  this  cross-correlation  test  program. 
They  were  on-site  according  to  the  schedule  shown  in  Table  1 1 -A.  Note 
that  Table  1 1 -A  also  outlines  the  data  used  when  all  participants' 
data  for  all  five  weeks  were  included  in  the  full  analysis.  Table  II- 
B  outlines  the  data  used  when  only  six  participants  and  only  four 
weeks'  data  were  included  in  the  analysis.  Table  II-B  shows  the  data 
used  in  the  “modified"  analysis  (without  Participant  7).  The  main 
body  of  this  report  deals  with  the  modified  analysis  dictated  by 
excessive  variability  found  in  the  data  from  Participant  7.  It  should 
be  noted  that  although  all  participants  are  named  in  Table  I,  they  are 
not  listed  in  the  same  order  as  Participant  Number.  This  was  done  to 
preserve  the  anonymity  of  the  data. 
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TABLE  I 

LIST  OF  TEST  PARTICIPANTS 


Pratt  &  Whitney  -  (E.  Hartford,  Connecticut) 

General  Electric  -  (Cincinnati,  Ohio) 

National  Gas  Turbine  Establishment  (NGTE)  -  (England) 

U.S.  Naval  Air  Propulsion  Center  (NAPC)  -  (Trenton,  New  Jersey) 
FAA  (MERF1)  -  (Atlantic  City,  New  Jersey) 

FAA  (MERF2)  -  (Atlantic  City,  New  Jersey) 

FAA  (Console)  -  (Atlantic  City,  New  Jersey) 


TABLE  I I— A 

PARTICIPANTS  INCLUDED  IN  FULL  ANALYSIS 
(All  Participants  /  5  Weeks'  Data) 


Participant: 

Week  1 
Week  2 
Week  3 
Week  4 
Week  5 


TABLE  I I— B 

PARTICIPANTS  INCLUDED  IN  MODIFIED  ANALYSIS 
( A1 1  Participants  Except  Number  7/4  Weeks1  Data ) 


Participant: 

Week  1 
Week  2 
Week  3 
Week  4 


to 


[*;< 


' "  -v 


mm. 


IV.  TEST  SETUP  AND  EQUIPMENT 


Emission  Source  and  Sampling  System 

A  mixed  flow  Pratt  and  Whitney  TF-30-P1  turbofan  engine  installed 
in  the  NAFEC  test  facility  was  used  as  the  emissions  source  for 
the  Phase  I  cross-correlation  program.  The  engine  was  modified 
by  removing  the  afterburner  assembly  and  installing  a  fixed  area 
exhaust  nozzle.  This  engine  incorporated  a  front  fan  having  a 
bypass- to-engine  airflow  ratio  of  approximately  1.09  to  1,  with 
the  fan  air  diverted  through  an  annular  duct  forming  the  outer 
shell  of  the  engine.  The  bypass  air  was  mixed  with  the  core 
airflow  downstream  of  the  turbine  and  the  mixed  flow  exited 
through  the  exhaust  nozzle.  The  compressor  pressure  ratio  is 
about  16  to  1,  and  the  engine  thrust  rating  was  11,500  pounds. 
Installation  of  the  engine  for  the  NAFEC  test  program  is  shown  in 
Fi  gure  1. 

Two  single-point  emissions  sampling  probes  were  installed  at  the 
exit  plane  of  the  exhaust  nozzle.  One  probe  was  located  two 
inches  from  the  engine  centerline  where  the  emissions  concentra¬ 
tions  were  high.  The  other  probe  was  located  ten  inches  from  the 
engine  centerline  in  an  area  of  low  concentration.  The  probes 
were  referred  to  as  the  "high"  and  "low"  probes.  Probe  locations 
are  shown  in  Figures  2  and  3. 

Each  probe  was  sized  to  provide  enough  sample  for  four  emissions 
systems  operating  simultaneously.  Only  one  probe  was  used  at  a 
time.  The  flow  from  the  probe  was  directed  into  a  heated  distri¬ 
bution  chamber.  Four  exit  ports  from  the  distribution  chamber 
provided  flow  to  the  four  instrumentation  systems  in  use  at  the 
same  time.  Any  excess  flow  was  spilled  overboard  at  the  distri¬ 
bution  chamber.  No  attempt  was  made  to  obtain  a  "representative 
sample".  The  objective  was  to  provide  a  steady  source  of  iden¬ 
tical  gas  samples  to  the  four  separate  measurement  systems.  A 
check  of  the  port-to-port  consistency  of  the  sample  was  made  by 
the  FAA  prior  to  the  cross-correlation  exercise  and  was  found  to 
be  satisfactory.  The  exit  ports  from  the  distribution  chamber 
were  numbered  from  1  through  4.  This  number  was  referred  to  as 
the  "port  number"  in  the  data  tabulations.  The  low  concentration 
probe  was  referred  to  as  "Probe  1"  and  the  high  concentration 
probe  was  "Probe  2". 

Each  participant  provided  his  own  heated  sample  line  from  the 
distribution  chamber  to  the  instruments.  Sample  lines  were 
heated  to  approximately  300°F  to  conform  with  existing  SAE 
practice  (SAE  ARP  1256)(5). 


B. 


Instrumentation 
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The  FAA  provided  three  separate  measurement  systems  used  in  the 
program.  The  first  was  a  stationary,  rack-mounted  gas  analysis 
system  referred  to  as  the  "Console".  The  other  two  were  trailer- 
mounted  systems  referred  to  as  "Mobile  Emissions  Research  Facili¬ 
ties"  (MERF1  and  MERF2)  (see  Figure  4).  Pratt  and  Whitney  had  a 
truck  mounted  system,  NGTE  used  a  trailer-mounted  system,  both  GE 
and  NAPC  had  rack-mounted  systems  which  were  moved  to  Atlantic 
City. 

Although  the  participants  used  gas  analyzers  made  by  different 
vendors,  the  operating  principle  was  the  same  in  all  cases.  The 
CO  and  CO?  instruments  were  non-di spersive  infrared  analyzers. 
The  total  hydrocarbon  analyzers  were  flame  ionization  detector 
(FID)  units.  The  N0X  analyzers  operated  on  the  chemiluminescent 
principle.  Thermal  converters  were  used  with  the  N0X  analyzers 
to  allow  both  NO  and  N0X  modes  of  operation. 

The  smoke  analyzers  were  all  built  and  operated  in  accordance 
with  SAE  ARP  1179(4)  except  for  the  presence  of  the  distribution 
chamber  and  used  the  stained  filter  paper  method.  The  filter 
paper  stains  were  then  measured  with  an  optical  reflectometer. 

Each  participant  used  his  own  computer  system  and  program  for 
reducing  the  raw  data.  The  final  values  for  gas  concentrations 
and  SAE  smoke  number  (SN)  were  then  reported  to  the  CRC  Test 
Coordinator  who  compiled  all  the  data. 


Span  Gases 


Each  participant  provided  his  own  zero  and  span  gases  for  the 
calibration  of  his  particular  analyzers.  In  addition,  a  set  of 
span  gases  was  provided  by  the  FAA  to  be  used  in  the  span  gas 
bias  tests.  These  gas  cylinders  were  supplied  and  analyzed  by 
Scott  Environmental  Technology. 


The  low  concentration  gases  were: 

CO  34.20  ppm 

C09  1.49  % 


11.97  ppm  C  (as  C3H8) 
9.36  ppm 


The  high  concentration  gases  were: 

CO  444.0  ppm 

COo  2.96  % 


174.3  ppm  C  (as  C3Hg) 
•107.0  ppm 


vi  ill 


All  gas  concentrations  listed  in  this  report  are  given  in  ppm  (by 
volume)  or  percent  (by  volume). 


Gas  mixtures  were  contained  in  cylinders  corresponding  high  power 
engine  operation  and  low  power  operation.  Four  different 
mixtures,  or  blends,  were  used.  The  four  blends  were:  (1)  high 
power  blend  of  CO,  C02,  and  THC;  (2)  low  power  blend  of  CO,  C02, 
and  THC;  (3)  high  power  concentration  of  NO;  and  (4)  low  power 
concentration  of  NO.  The  CO,  C02,  and  THC  blends  had  balance 
air;  the  NO  blend  was  balance  N2.  In  addition,  four  identical 
cylinders  of  each  blend  were  purchased  so  that  identical  concen¬ 
tration  gases  could  be  used  throughout  the  program.  All  four 
participants  were  able  to  analyze  the  same  gas  simultaneously. 


V.  TEST  PROCEDURE 


During  the  planning  process  it  was  agreed  that jt  was  not  practical  to 
have  all  the  participants  at  Atlantic  City  at  the  same  time,  but  that 
four  participants  at  a  time  was  a  manageable  number.  A  testing 
schedule  was  established  covering  a  period  of  five  weeks.  A  staggered 
arrangement  was  used  in  which  each  of  the  four  visiting  participants 
was  scheduled  for  two  consecutive  weeks.  The  scheduling  arrangement 
along  with  the  sampling  port  assignments  are  shown  in  Table  III. 


TABLE  III 

SAHPLIN6  PORT  ASSIGNMENTS  BY  PARTICIPANT 


Sampling  Port  Number 
Week  Test  Week  Date 


Participants 
12  3  4 


1  10-24-78 

2  10-31-78 

3  11-7-78 

4  11-14-78 

5  3-14-79 


3  2  4  1 
3  5  4  1 
3  5  6  1 
3  7  6  1 
3  7  2  1 


The  NAFEC  Console  (Participant  3)  and  MERF1  (Participant  1)  were  on 
test  during  all  five  weeks.  J1ERF2  (Participant  2)  was  used  during 
weeks  1  and  5  to  fill  out  the  schedule,  and  thus  provide  four  partici¬ 
pants  for  each  of  the  five  weeks. 
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A  test  sequence  was  established  for  a  one-week  period  in  which  the 
engine  was  run  at  a  number  of  power  settings  (Modes  1  through  6)  to 
provide  a  range  in  emission  concentrations.  The  same  sequence  was 
repeated  for  each  of  the  five  weeks.  The  order  of  power  settings  was 
changed  within  the  different  sequences  to  provide  information  on 
emission  measurement  hysteresis  although  not  a  major  test  objective 
and  not  analyzed  in  this  report.  The  weekly  test  sequence  is  shown  in 
Table  IV. 


TABLE  IV 
TEST  SEQUENCE 


Probe 

High  Concentration. 

Low  Concentration 

Sequence 

A 

B 

C 

D 

E 

Measurements 

BBSS! 

Gaseous 

Gaseous 

Gaseous 

Smoke 

Engine  Modes 

i 

1 

1 

1 

1 

2 

6* 

2 

6* 

2 

•  3 

5 

3 

5 

3 

4 

4 

4 

4 

5* 

5 

3 

5 

3 

3 

6* 

2 

6* 

2 

2 

5 

1 

5 

1 

1 

4 

2 

4 

2 

5* 

3 

3 

3 

3 

3 

2 

4 

2 

4 

2 

1 

5 

1 

5 

1 

6* 

6* 

2 

1 

1 

2 

2 

Where:  Mode  Nominal  Power 


1 

2 

3 

4 

5 

6 


Idle 

High  Idle 

Approach 

Cruise 

Max  Continuous 
Take-off 


No  emissions  readings  taken  at  these  points. 


To  provide  consistent  measurements,  the  engine  was  allowed  to 
stabilize  before  taking  measurements.  The  stabilization  times  were  20 
minutes  for  initial  start  to  idle,  10  minutes  for  subsequent  idle 
settings,  and  5  minutes  for  all  other  engine  mode  settings.  Emission 
measurements  were  not  made  at  the  take-off  power  setting  (Mode  6)  due 
to  engine  run  time  limitations. 

The  original  test  sequence  plan  called  for  smoke  readings  at  high 
power  setting.  During  the  first  week  of  testing  it  was  discovered 
that  this  gave  very  high  smoke  numbers  which  are  no  longer  encountered 
in  current  low  smoke  technology  engines.  Consequently,  the  smoke 
sequence  was  revised  as  shown  in  Table  IV,  beginning  with  the  second 
week.  Smoke  readings  were  then  taken  only  at  idle,  high  idle,  and 
approach  modes  with  the  low  concentration  probe. 

The  weekly  test  sequence  also  included  a  pretest  span  gas  bias  check 
which  was  run  just  prior  to  the  engine  emission  test,  and  a  post  test 
span  gas  bias  check  which  was  run  just  after  the  engine  gaseous  emis¬ 
sion  test  and  before  the  smoke  test.  During  the  span  gas  bias  test 
all  eight  of  the  special  span  gas  concentrations  were  checked,  and 
each  participant  took  four  replicate  readings  of  each  gas. 


VI.  TEST  DATA 


The  test  data  used  in  this  analysis  is  reproduced  in  the  appendices  as 
follows: 


•  Appendix  B  -  Calibration  Gas  Measurement  Summary 

This  appendix  contains  all  the  emissions  span  gas  results  for  the 
four  gases:  CO,  COg.  THC,  and  N0X.  The  levels  reported  are  the 
average  of  four  readings  for  each  participant  and  are  accompanied 
by  an  estimate  of  each  participant's  standard  deviation,  IS. 
These  tables  also  list  each  test  day's  average  across  partici¬ 
pants  for  each  span  gas,  X,  along  with  an  estimate  of  daily 
participant-to-participant  variabi  1 i ty. 


•  Appendix  C  -  Gaseous  Exhaust  Measurement  Data 

This  appendix  contains  all  the  emissions  data  analyzed  for  this 
report.  It  is  the  engineering  unit  data  from  each  participant's 
instruments.  It  is  the  intent  of  this  analysis  to  utilize,  in 
the  final  results,  each  participant’s  test  data  without  cor¬ 
recting  them  to  the  calibration  gas  standards  used  at  the  test 
site  to  evaluate  each  participant's  calibration  gases.  These 
results  would  reflect  what  would  normally  result  from  inter- 


facility  comparisons  where  no  standard  gas  is  exchanged  for 
corrections.  The  exception  was  to  allow  corrections  to  NGTE  data 
due  to  their  analysis  of  their  calibration  gases(6)  which  had  to 
be  done  after  the  test;  every  other  participant  had  time  to 
complete  that  analysis  before  the  test. 


•  Appendix  D  -  Smoke  Data 

This  appendix  contains  all  the  smoke  data  available  for  this 
analysis.  Gnly  the  data  from  the  low  concentration  probe  at 
power  levels  of  approach  and  below  were  analyzed  to  try  to  screen 
out  some  of  the  very  high  smoke  numbers  obtained  from  the  engine, 
but  not  typical  of  current  low  smoke  technology.  Even  with  that 
screening,  however,  smoke  results  are  of  doubtful  current  value 
as  the  lowest  smoke  number  in  all  the  data  is  greater  than  20. 


A  complete,  detailed  tabulation  of  all  the  test  data  from  this  program 
is  on  file  in  the  CRC  compendium  cited  as  Reference  1. 


VII.  DATA  ANALYSIS 


This  section  will  outline  the  analysis  performed  to  determine 
partici pant-to-partici pant  variability  on  clean  span  gases, 
participant-to-participant  variability  on  exhaust  emissions,  indivi¬ 
dual  participant  precision  errors,  and  the  bias  introduced  by  span 
gases  along  with  the  results  of  an  attempt  to  remove  that  bias.  The 
smoke  data  analysis  is  also  discussed.  Detailed  definitions  of 
precision  and  bias  may  be  found  in  the  ASME  Test  Code  Supplement*7 ) . 


A.  Participant-to-Participant  Variability  on  Clean  Span  Gases 

Each  participant  measured  the  gas  concentration  levels  of  gas 
mixtures  intended  to  simulate  low-  and  high-power  engine  emis¬ 
sions.  The  span  gas  vendor  analysis  results  were  as  shown  in 
Section  IV-C. 

Each  participant  analyzed  each  gas  four  times  on  each  of  two  days 
during  each  week  they  were  present  at  the  test  site.  The  average 
values  for  each  of  those  sets  of  four  are  shown  in  Appendix  B, 
along  with  each  participant's  IS  variability. 


Setting  each 
Xijk 

where:  1 

j 


k 


span  gas  measurement  as  'X',  we  have: 

=  individual  gas  measurement 

*  1  to  4,  the  number  of  repeat  readings  on  one  gas 

=  1  to  10,  the  number  of  the  test  day,  2  days  each 

of  5  weeks 

=  1  to  4,  the  number  of  participants  present 

during  any  one  day 


We,  therefore,  have  an  array  of  averages  and  standard  deviations, 
given  in  Appendix  B,  of: 


Xjk  .  Xj,4 

Sjk  .  Sj,4 


X10,k  ••••  x10,4 
S10,k  ••••  S10,4 


where: 


jk 

l 

1=1 


the  average  span  gas  reading 
of  participant  k  on  day  j. 


Njk 


I 

i  =  l 


y  \Z 


<Xijk  -  Xjk) 


1/2 


Njk  - 


the  span  gas  reading 
standard  deviation  of 
participant  k  on  day  j 


Njk  =  4  (usually),  the  number  of  repeat  span 

gas  readings  by  each  participant. 


The  Xju  and  are  listed  for  each  participant  and  span  gas  in 
Appendix  B  ana  are  listed  under  the  general  heading,  Avg.  +1S. 


To  determine  the  participant-to-participant  variability,  a  new 
array  is  created: 


2S< 


X10  2S10 


where: 


J  — 

£  X jk/NJ 
k=l 


;  the  average  span  gas  reading  of 
all  participants  on  day  j. 


I 

k=l 


the  standard  deviation 
of  span  gas  readings  by 
all  participants  on  day  j. 


f.'.j  =  4  (usually),  the  number  of  participants 

on  day  j. 


From  the  above  array,  we  calculate: 


=  L  L 

X  =  £  (Xj)  (Nj)/  2lNj  •,  the  grand  average  span  gas 

j=l  j=l  J  reading  of  all  participants 

on  all  days. 


I  <2S,)2  (Nrl) 
j=l  J  J 
L 

L  (Nrn 

j=i  J 


1/2 


>  ;  the  pooled  standard 

deviation  of  span  gas 
readings  for  all  parti - 
/  cipants  across  days. 


L  =10  for  5  weeks  of  data  and 
8  for  4  weeks  of  data. 


The  Xj  values  are  in  the  third  column  from  the  right  in  Appendix 
B  andJare  listed  under  the  heading,  X. 


The  2Sj  values  are  shown  in  the  extreme  right  columns  of  the 
tables  in  Appendix  B  and  are  listed  under  the  heading,  2SX. 

The  2$_  values  are  the  participant-to-participant  variability  and 
are  presented  for  each  constituent  and  level,  X,  along  with  their 
degrees  of  freedom  (d.f.)  in  Tables  V-A  and  V-B.  Table  V-A  is 
for  all  participants,  5  weeks  data.  Table  V-B  is  for  only  4 
weeks  data  and  does  not  include  Participant  7.  Table  V-B 
presents  the  final  results  of  the  span  gas  comparisons  intended 
to  yield  participant-to-participant  variability  on  clean  span 
gases. 

Note  that  no  attempt  has  been  made  to  remove  from  Tables  V-A  or 
V-B  the  contribution  due  to  each  participant's  precision  on  span 
gases,  S^k.  Removing  it  would  have  negligible  impact  on  the 
presented  results.  Note  too  that  although  some  Participant  7 
values  shown  in  Tables  B-I  through  B-IV  were  obvious  outliers, 
they  were  not  omitted  from  the  calculation  results  shown  in 
Table  V-A  so  the  effect  of  including  Participant  7  could  more 
clearly  be  shown. 


TABLE  V-A 

PARTICIPANT-TO-PARTICIPANT  CALIBRATION  VARIABILITY  (AVG.  t 2S) 
(All  Participants  /  5  Weeks'  Data) 


CO,  ppm 

Calibration 

Low 

34.2 

Gas  Concen. 

High 

444.0 

Variability 

-  Low 

35.7+  9.5 

(Avg.  +2S) 

-  High 

448.2+25.4 

±2S/Avq.-%;  d 

l.f. 

Low 

+26 . 6 ; 26 

High 

+  5.7 ; 28 

Constituent 


co2,  % 

THC,  ppmC 

N0„,  ppm 

1.49 

11.97 

9.36 

2.96 

174.3 

107.0 

1.49+0.074 

16.1+18.3 

8.97+2.3 

2.96+0.158 

171.0+68.9 

106.6+18.0 

+5.0; 29 

+113.7 ; 28 

+25.6;28 

+5. 3;29 

+  40 . 3 ; 29 

+16.9;29 

Note:  d.f.  =  Degrees  of  Freedom 
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TABLE  V-B 

PARTICIPANT-TO-PARTICIPANT  CALIBRATION  VARIABILITY  (AV6.  ±2S) 
(All  Participants  Except  Number  7/4  Weeks'  Data) 


Constituent 


CO,  ppm 

C0?,  % 

THC,  ppmC 

N0„,  ppm 

Cal ibration 

Low 

34.2 

1.49 

11.97 

9.36 

Gas  Concen. 

High 

444.0 

2.96 

174.3 

107.0 

Variability 

-  Low 

34.1+  3.6 

1.48+0.057 

14.7+11.2 

9.22+0.78 

(Avg.  +2S) 

-  High 

444.6+16.9 

2.94+0.091 

178.8+16.2 

109.0+6.1 

±2S/Avq.-%; 

d.f. 

Low 

+10.6; 19 

+3 . 9 ; 21 

+7  6.2; 22 

+8 . 5 ; 21 

High 

+  3 . 8 ; 22 

+3. 1;21 

+  9.1; 22 

+5.6;22 

Note:  d.f.  =  Degrees  of  Freedom 


B.  Participant-to-Participant  Variability  on  Exhaust  Gases 

Each  participant  measured  exhaust  gases  simultaneously  (alter¬ 
nating  between  a  high  and  low  concentration  probe)  when  the 
engine  stabilized  at  each  of  five  power  levels  (modes).  All  the 
test  data  secured  are  shown  in  Appendix  C,  Gaseous  Exhaust 
Measurement  Data.  To  proceed  with  this  analysis,  probe/mode 
combinations  were  chosen  for  each  gas  at  each  of  two  concen¬ 
tration  levels  which  approximated  the  levels  of  the  span  gases. 
There  were  three  to  five  data  points  per  probe/mode  combination 
for  each  of  the  four  participants. 

For  those  data  chosen,  the  participant-to-participant  variability 
and  the  individual  participant  precision  were  determined  by 
Grubbs'  method15’.  This  was  the  same  method  used  by  Wedlock(6’ 
in  his  partial  analysis  of  the  data.  This  analysis  method  yields 
estimates  of: 

a.  Individual  participant  precision; 

b.  Participant-to-participant  variability  which  includes 
the  effect  of  "a"  above; 

c.  Engine  variability,  the  repeatability  of  setting  the 
engine  conditions. 


£1 


Participant-to-participant  variability  was  determined  by  the 
Grubbs'  method  as  follows: 


Let  Yijk 


an  exhaust  gas  measurement  during  a  given  week 


where: 


1  to  4,  the  number  of  the  participant  sampling 
a  data  point 


j  =  1  to  4,  the  number  of  the  probe/mode  combina¬ 

tion  approximating  the  calibration  gas  levels 
used 


k  =  1  to  5,  the  number  of  the  data  or  sample  point 

matching  each  probe/mode  combination,  j 


The  variability  between  each  participant  on  a  data  point  is  then 
computed  as: 


2S 


pjk 


( Yi jk  ‘  Yjk 

TT 


I  1/2 


where:  Yjk  = 


Yijk/1 


and 


I  =  4,  usually,  the  number  of  participants 
sampling  a  data  point 


Then  the  2SD]-k  values  are  pooled  across  the  points  sampled  during 
a  given  weexat  the  probe/mode  combination  chosen: 


2Sp(pool )j 


where:  K  =  3  to  5,  the  number  of  sample  points  matching 

the  jth  probe/mode  combination  chosen 


2 

k=l 


<2Spjk)2 


L/C 


The  Grubbs'  calculation  yields  the  individual  participant 
precision,  S^,  for  a  given  probe/mode  combination.  The  details 
of  this  calculation  are  given  in  Reference  5  and  will  not  be 
repeated  here.  The  S.,-  values  are  pooled  using  the  following 
relation: 


4 

3a 

»vl 

4! 


2S  =  2 


J^r1. 


where:  =  Individual  participant  precision  for  each 

participant,  i 

K.j  =  3  to  5,  the  number  of  test  points  used  to 

calculate  Si 

I  =  the  number  of  participants  (usually  4) 


Partici pant-to-parti ci pant  precision  and  bias  are  bound  up  in 
2Sp( poplli  and  are  not  separable.  The  effect  of  individual 
participant  precision  on  participant-to-parti cipant  variability 
is  removed  by  the  following  calculation: 


j)2  -  (2?)2] 


Jp(pool )j 


where:  2Spp  =  Participant-to-participant  variability 

The  individual  participant  precision  is  usually  small  relative  to 
the  participant-to-participant  variability.  (2S  )  is  the 
participant-to-participant  variability  for  a  given^gas,  for  a 
given  probe/mode  combination,  and  for  one  week  only.  These 
values  are  pooled  over  all  4  or  5  weeks,  as  required,  to  give: 


2SPP* 


Z  [(2Spp)2(dfm)] 
m-  I 

M 

m=l 


where:  dfm  =  number  of  degrees  of  freedom  for  the  mth  week 

M  =  4  or  5,  the  number  of  weeks  included 


(2Spp*)  is  the  overall  participant-to-participant  variability  for 
one^given  probe/mode  combination. 

These  values  are  pooled  across  the  probe/mode  combinations  chosen 
for  the  particular  gas  and  emission  level  (see  Table  VII)  to 
yield: 


.’vy*vi 


where:  J  =  2  to  4,  the  number  of  probe/mode  combina¬ 

tions  used. 


The  detailed  calculations  performed  using  Grubbs'  method(5)  are 
on  file  in  the  CRC  office. 

The  participant-to-participant  variability  2SpD**  is  presented  in 
Tables  VI-A  and  VI-B,  in  which  each  2S_  **  f$  shown  along  with 
the  average  levels.  (Degrees  of  freedom  were  always  30  or  above, 
so  2S  **  is  appropriate).  Table  VI-A  is  for  all  seven  partici¬ 
pants  and  five  weeks  data.  Table  VI-B  is  for  six  participants 
and  four  weeks  data.  Participant  Number  7  was  excluded  as  being 
much  too  erratic. 


Table  VI-B  represents  the  final  results  of  the  study;  it 
describes  the  expected  variability  between  laboratories  testing 
the  same  engine  at  the  same  time. 

The  probe/mode  combinations  used  in  calculating  Tables  VI-A  and 
VI-B  are  shown  in  Table  VII. 


TABLE  VI-A 

PARTICIPANT-TO-PARTICIPART  TEST  VARIABILITY  (AVG.  ±2S) 
(All  Participants  /  5  Weeks'  Data) 


Consti tuent 


CO,  ppm 

co2,  % 

THC,  ppmC 

N0V,  ppm 

Concentration* 

(Avg.  +2S) 

Low 

35.4+14.4 

1.67+0.15 

17.7+  8.2 

13.4+4.7 

High 

505.9+20.4 

2.66+0.27 

204.0+110.2 

81.1+7.6 

+2S/Avg.  -  % 

Low 

+40.6 

+  9.2 

+46.6 

+35.2 

Hi  gh 

+  4.0 

+  10.0 

+54.0 

+  9.4 

*  Concentrations 
All  degrees  of 

chosen  to  approximate 
freedom  over  30. 

calibration  gas 

levels. 
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TABLE  VI-B 

PARTICIPANT-TO-PARTICIPANT  TEST  VARIABILITY  (AV6.  ±2S 


(All  Participants  Except  Number  7/4  Weeks'  Data) 


Concentration* 
(Avg.  +2S) 

Low 

High 

+2S/Avg.  -  % 
Low 
High 


CO,  ppm 


32.4+  8.1 
496.6+18.6 


+24.9 
+  3.7 


Constituent _ 

COg,  %  THC,  ppmC  N0X>  ppm 


1.68+0.11  17.9+2.9  14.1+1.7 

2.66+0.14  210.6+15.8  82.0+5.0 


+16.0 
+  7.5 


+11.9 

+  6.1 


*  Concentrations  chosen  to  approximate  calibration  gas  levels. 
All  degrees  of  freedom  over  30. 


TABLE  VII 

PROBE/MODE  *  COMBINATIONS  OF  THE  DATA  INCLUDED 


IN  TABLES  VI-A,  VI- 

B,  AND  X 

Constituent 

CO 

C0? 

THC 

N0V 

Concentration 

Low 

1/4, 1/5, 
2/5 

1/4, 1/5, 
2/1, 2/2 

1/3, 2/3 

1/2, 1/3 
2/1, 2/2 

High 

2/1, 2/2 

2/4, 2/5 

2/1, 2/2 

2/4, 2/5 

*  Note: 

Probe 

Code 

Mode  Code 

1  =  Id! 

2  =  Hig 

3  =  App 

4  =  Cru 

5  =  Max 
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Individual  Participant  Precision  Errors 


The  Grubbs  analysi s' 3 ' ,  as  mentioned  above,  yields  individual 
participant  precision  errors.  Those  precision  errors,  appro¬ 
priately  pooled  over  the  same  probe/mode  data  points  given  in 
Table  VII,  are  shown  in  Table  VIII  along  with  their  average 
values.  Table  VIII  and  all  subsequent  tables  include  four  weeks' 
data  and  do  not  include  Participant  Number  7. 

The  Fortran  program  utilized  to  perform  the  Grubbs'  analysis(5) 
was  obtained  from  NGTE  and  run  at  General  Electric,  Cincinnati. 
A  copy  of  the  Fortran  listing  is  at  the  CRC  office  for  reference. 


TABLE  VIII 

INDIVIDUAL  PARTICIPANT  TEST  DATA  PRECISION 

(Average  Concentration  +2S),  Low  and  High 
(2S/Average  in  Percent),  Low  and  High 


Participant 


Constituent _ 

CQg,  %  THC,  ppmC  NQy,  ppm 


Low 

High 

Low  % 
High  % 

32.5+1.2 

497.1+5.4 

+3.7 

+1.1 

1.68+0.05 

2.66+0.07 

+2.8 

+2.7 

18.3+2.1 
213.5+8.3 
+11.3 
+  3.9 

14.1+0.8 

82.0+2.7 

+5.7 

+3.3 

Low 

High 

Low  % 
High  % 

30.1+  2.2 
490.0+18.1 
+7.4 
+3.7 

1.69+0.02 
2.64+0.03 
+  1.3 
+  1.2 

17.0+3.5 
203.0+9.9 
+20.6 
+  4.9 

13.9+3.0 
83.2+7.3 
+21.5 
+  8.8 

Low 

High 

Low  % 
High  % 

32.5+1.3 

497.1+3.9 

+3.9 

+0.8 

1.68+0.14 

2.66+0.14 

+8.4 

+5.4 

18.0+2.2 
210.9+5.4 
+12.2 
+  2.5 

14.1+1.2 
82.0+1.4 
+8.3 
+  1.7 

Low 

High 

Low  % 
High  X 

30.5+1.9 

492.3+4.9 

+671 

+1.0 

1.69+0.06 

2.65+0.06 

+3.6 

+2.4 

17.0+2.8 
198.8+7.9 
+  16.4 
+  4.0 

14.2+0.8 

83.1+4.3 

+5.7 

+5.2 

Low 

High 

Low  % 
High  X 

32.7+4.4 
496.5+5.4 
+13.4 
+  1.1 

1.67+0.05 

2.66+0.11 

+3.0 

+4.2 

17.0+1.0 

208.8+3.1 

+5.7 

+1.5 

14.1+1.4 

81.1+2.8 

+9.7 

+3.4 

Low 

High 

Low  % 
High  % 

34.5+1.8 

501.9+6.7 

+5.3 

+1.3 

1.67+0.10 

2.67+0.15 

+6.1 

+5.6 

19.0+  3.8 
222.9+14.6 
+20.2 
+  6.6 

14.0+0.9 

80.9+2.6 

+6.1 

+3.2 

All  degrees  of  freedom  over  30. 
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Bias  Introduced  by  Each  Participant's  Span  Gases 

From  the  data  in  Appendix  B,  an  estimate  of  the  contribution  of 
span  gas  bias  to  participant-to-partici pant  variability  may  be 
calculated.  This  is  done  by  comparing  each  participant's  average 
value  for  the  span  gas  with  that  obtained  on  that  day  by  a 
referee  system,  the  FAA  Console.  These  calibration  (or  span)  gas 
correction  factors  are  given  in  Table  IX  where  the  Console 
average  is  divided  by  the  participant's  average.  The  gas  concen¬ 
trations  are  the  analysis  levels  provided  by  Scott. 


TABLE  IX 

CALIBRATION  SAS  CORRECTION  FACTORS 
CONSOLE  DATA/PARTICIPANTS  DATA 
(All  Participants  Except  Nunber  7/4  Weeks'  Data) 


Participant  : 


Gas  (^conc.) 


CO  (34.2  ppm) 

0.975 

0.9881 

0.9549 

0.9374 

0.8743 

CC  (444.0  ppm) 

1.023 

0.9934 

0.9963 

0.9879 

0.9739 

C02  (1.49%) 

1.016 

1.000 

1.000 

0.9933 

0.9850 

C02  (2.96%) 

0.997 

0.9915 

1.006 

1.02C 

1.012 

THC  (11.97  ppmc) 

G.  522 

0.5681 

0.8773 

0.8798 

0.9991 

THC  (174.3  ppmc) 

0.912 

0.9233 

0.9731 

0.9952 

0.9733 

N0X  (9.36  ppm) 

0.990 

0.9876 

0.9736 

1.009 

0.9204 

N0X  (107.0  ppm) 

0.978 

1.033 

0.9989 

0.9592 

0.9657 

The  original  data  given  in  Appendix  C  were  multiplied  by  the 
appropriate  factors  from  Table  IX  to  give  the  "calibration  gas 
corrected"  data.  These  data  were  then  used  to  compute  Table  X. 
The  "calibration  gas  corrected"  data  show  no  significant  improve¬ 
ment  over  the  "uncorrected"  data  in  Table  VI-B. 


TABLE  X 

PARTICIPANT-TO-PARTICIPANT  TEST  VARIABILITY  (AV6.  ±  2S 
CALIBRATION  6AS  CORRECTED 
(All  Participants  Except  Nunber  7/4  Meeks'  Data) 


Constituent 


Concentration* 
(Avg.  +2S) 


Low 

31.0+  5.4 

1.68+0.10 

14.7+  9.0 

13.8+1.8 

High 

496.5+12.4 

2.67+0.17 

203.4+20.5 

80.8+3.6 

+2S/Avg.  -  % 

Low 

+  17.5 

+5.9 

+60.9 

+13.0 

High 

+  2.5 

+6.4 

+  10.1 

+  4.5 

*  Concentration  chosen  to  approximate  calibration  gas  levels. 
All  degrees  of  freedom  are  over  30. 


Table  VI-B  is  the  final  result  of  this  analysis.  It  does  not 
include  the  effect  of  individual  precision  errors.  It  does 
however  include  any  span  gas  caused  differences  which  would  be 
difficult  to  evaluate  when  comparing  test  results  from  separate 
facilities. 


The  smoke  data  utilized  is  presented  in  Appendix  D.  The  results 
of  the  brief  regression  analysis  of  the  smoke  data  are  presented 
in  Table  XI.  These  smoke  data  were  insufficient  to  provide 
adequate  determination  of  present  day  participant-to-participant 
variability. 


TABLE  XI 

REGRESSION  ANALYSIS  OF  SMOKE  MEASUREMENTS 
(All  Data  Referenced  to  FAA  Console) 


Parti¬ 

cipant 

No.  of 
Data 
Points 

Slope 

Intercept 

SN 

Correlation 

Coefficient 

Standard 
Error  of 
Estimate 
SN 

SN  at 
Refer, 
of  30 

1 

40 

1.098 

1.17 

0.958 

4.32 

34.1 

4 

9 

1.108 

-1.83 

0.970 

3.27 

31.4 

5 

16 

1.166 

-2.64 

0.983 

2.76 

32.3 

6 

20 

1.045 

2.46 

0.987 

2.17 

33.8 

7 

10 

1.129 

8.89 

0.986 

2.40 

42.8 

F.  Outlier  Rejection 


In  all  the  above  analysis,  the  Grubbs  outlier  rejection  method  of 
Reference  8  was  employed.  It  was  used  as  the  basis  for  rejecting 
individual  points  and  for  rejecting  one  participant. 


VIII.  RESULTS  AMD  DISCUSSION 


It  is  the  intent  of  this  section  to  present  and  discuss  the  analysis 
results  from  two  perspectives:  the  meeting  of  test  objectives,  and 
other  comparisons  of  interest. 


A.  Meeting  of  Test  Objectives 

•  Objective  1  -  "Establish  a  Cross-Correlation" 

This  objective  was  met  with  completion  of  the  testing  at 
NAFEC. 


•  Objective  2  -  "Determine  Typical  Variability  Between  Testing 

Groups" 

This  objective  is  met  by  the  presentation  of  Table  VI-B  and 
Table  XI.  Table  VI-B  provides  estimates  of  the  variability 
that  can  be  encountered  between  testing  groups  measuring 
gaseous  emissions  on  the  same  turbine  engine.  The  2S  vari¬ 
ability  shown  does  not  include  the  effect  of  individual 
participant  precision  but  does  include  the  effect  of  dif¬ 
ference  in  each  participant's  calibration  gases.  In 
applying  the  data  in  Table  VI-B  to  future  comparisons,  the 
effect  of  each  participant's  precision  needs  to  be  worked 
into  the  analysis.  This  usually  can  be  done  after  evalu¬ 
ating  the  precision  errors  during  test.  The  effect  of 
calibration  gas  differences  was  left  in  the  results  in  Table 
VI-B  because  future  testing  would  likely  not  provide  for 
easy  comparisons  of  calibration  gases.  The  calibration  gas 
differences  are,  therefore,  taken  as  typical  for  future 
tests  as  well. 

It  should  be  noted  that  Table  VI-B  does  not  include  Par¬ 
ticipant  Number  7  or  Week  5  data.  A  comparison  of  Tables 
V I -A  (which  includes  Participant  7)  and  VI-B  illustrates 
clearly  why  the  decision  to  eliminate  Participant  Number  7 
was  made.  As  a  participant.  Number  7  was  an  outlier;  it  was 
therefore  eliminated  from  all  further  analysis. 


Note  also  that  while  the  EPA  has  a  +2%  specification  on 
calibration  gases,  analyzer  drift,  etc.,  there  should  be  the 
awareness  that  participant-to-participant  variability  is  an 
accumulation  of  those  errors  plus  errors  from  other  sources. 
The  result  can  be  variability  greatly  in  excess  of  +2%,  as 
the  data  in  Table  VI-B  show. 

In  addition,  the  results  in  Table  VI-B  were  obtained  under 
carefully  controlled  conditions  such  as  existed  at  NAFEC  for 
this  comparison.  Less  controlled  conditions  would  probably 
yield  increased  variability. 

Table  XI  presents  the  results  of  the  regression  analysis  of 
smoke  measurements.  A  simple  regression  analysis  was  used 
for  smoke  since  there  were  not  enough  data  to  perform  the 
Grubbs'  analysis(5)  used  for  the  gaseous  emissions.  While  a 
direct  comparison  is  not  possible,  the  low  correlation 
coefficients  and  high  standard  errors  of  estimate  for  smoke 
show  that  the  variability  for  smoke  is  considerably  greater 
than  that  for  the  gaseous  emissions.  This  points  out  the 
fact  that  of  the  various  measurements  made  under  this 
program,  smoke  measurement  is  the  area  most  in  need  of 
improvement. 

It  should  be  noted  that  in  this  report,  the  smoke  measure¬ 
ments  were  in  the  SN  range  from  20  to  80.  This  range  is  too 
high  to  be  applicable  to  current  engine  technology  where 
smoke  numbers  are  usually  20  or  lower. 


Objective  3  -  "Estimate  Participant  Precision  Errors  and 

Evaluate  Their  Effect  on  Observed  Variability" 

This  objective  was  met  by  the  presentation  of  Appendix  A  and 
Table  VIII.  Appendix  A  presents  individual  participant 
precision  (IS)  on  clean  span  gases.  By  inspection,  it  is 
shown  that  these  IS  values  have  a  negligible  effect  on  span 
gas  comparisons.  Table  VIII  presents  the  individual  parti¬ 
cipant  precision  errors  for  each  participant  as  revealed  by 
measuring  test  gases.  These  data  are  much  more  represen¬ 
tative  of  errors  that  effect  test  comparisons  than  are  the 
data  in  Appendix  A.  The  effect  of  the  precision  errors  in 
Table  VIII  is  not  included  in  the  results  in  Table  VI-B; 
however,  upon  inspection,  they  are  seen  to  have  measurable 
but  small  effects.  The  major  source  of  the  variability  in 
Table  VI-B  was  participant-to-participant  variability,  not 
individual  participant  precision. 


•  Objective  4  -  "To  Evaluate  the  Bias  Introduced  by  Span 

Gases" 

This  objective  was  met  with  the  presentation  of  Table  IX, 
"Calibration  Gas  Correction  Factors."  These  factors  were 
determined  by  choosing  Participant  3  (FAA  Console)  as  a 
referee  present  for  all  testing,  and  comparing  all  partici¬ 
pants'  calibrations  to  that.  The  average  of  all  the  correc¬ 
tion  factors  (eliminating  the  two  THC  outliers  of  0.522  and 
0.5681)  is  0.9765,  and  the  2S  is  +0.0798  (+8.2%).  The 
scatter  (2S)  in  these  correction  factors  should  be  represen¬ 
tative  of  participant-to-participant  calibration  gas  scatter 
(plus  a  negligible  instrument  precision  contribution).  If 
all  the  participant  span  gases  were  within  the  recommended 
accuracy  of  +2%,  the  correction  factor  2S  would  be  +2  xVZ 
or  +2.8%  (scatter  of  differences,  or  ratios,  between  two 
participants).  Since  the  observed  scatter  is  +8.2%,  there 
is  the  suggestion  that  the  individual  span  gases  of  the 
participants  are  only  good  to  +8.2  *  V2  or  +5.8%.  This  is 
more  than  eight  times  the  permitted  variability  of  +2%  (5.82 
t  22).  Apparently,  even  on  clean  span  gases,  participants 
can’t  agree  within  EPA  guidelines. 


Other  Comparisons  of  Interest 

Table  X  presents  the  participant-to-participant  variability 
determined  after  correcting  the  data  sets  by  the  calibration  gas 
factors  of  Table  IX.  A  comparison  of  Table  X  with  VI-B  shows 
that  the  calibration  gas  corrections  had  little  effect  on 
participant-to-participant  variability.  The  variability  is  due 
to  some  other  major  system-to-system  error  source  or  sources 
which  have  not  been  identified. 

Tables  XII-A  and  XII-B  present  the  pooled  participant  test  pre¬ 
cision  as  a  percent  of  level.  Table  XII-A  is  for  all  partici¬ 
pants,  five  weeks'  data;  and  Table  XII-B  all  but  Participant  7, 
four  weeks'  data.  The  data  in  Tables  XII-A  and  XII-B  are  shown 
in  bargraphs  in  Figures  5A  and  5B.  Note  that  with  the  elimina¬ 
tion  of  Participant  7,  the  data  becomes  well  behaved.  It  shows 
that  participant  test  precision  as  a  percent  of  level  decreases 
with  increasing  concentration.  The  results  shown  in  Figure  5 
also  contributed  to  the  rationale  for  eliminating  Participant  7 
as  an  outlier. 
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TABLE  XII-A 

BAR6RAPH  DATA  FOR  POOLED  PARTICIPANT  TEST  PRECISION 


Gas 


(All  Participants  /  5  Weeks) 


CO 


Probe/Mode 

Concentration 

1/5 

1/4 

2/5 

2/4 

1/3 

1/1 

1/2 

2/3 

2/2 

2/1 

(ppm  CO) 

27.5 

35.0 

43.6 

57.9 

84.0 

147.2 

147.3 

178.2 

485.9 

525.9 

+2S/Avg.  % 

50.0 

43.9 

32.0 

40.9 

23.1 

10.6 

11.1 

12.3 

4.1 

4.0 

Gas 

CO 

2 

Probe/Mode 

Concentration 

1/1 

1/2 

1/3 

1/4 

1/5 

2/1 

2/2 

2/3 

2/4 

2/5 

(%  COo) 

0.58 

0.68 

0.98 

1.47 

1.69 

1.73 

1.79 

1.98 

2.53 

:  2.79 

+2S/Avg.  % 

12.5 

12.0 

9.7 

7.2 

6.2 

10.6 

11.0 

10.6 

10.8 

9.3 

Gas 

THC 

Probe/Mode 

Concentration 

2/5 

1/5 

2/4 

1/4 

1/3 

2/3 

1/2 

1/1 

2/2 

2/1 

(ppmC  THC) 

3.6 

4.0 

4.1 

4.1 

11.7 

23.6 

40.8 

52.3 

185.8 

222.1 

+2S/Avg.  % 

109.8 

56.8 

98.5 

54.3 

48.7 

42.9 

58.8 

57.7 

52.7 

54.6 

Gas 

NO 

V 

Probe/Mode 

Concentration 

1/1 

1/2 

1/3 

2/1 

2/2 

2/3 

1/4 

1/5 

2/4 

2/5 

(ppm  NO  ) 
+2S/Avg.  % 

4.8 

6.4 

14.9 

15.0 

17.2 

31.8 

37.8 

52.5 

70.1 

92.0 

33.8 

32.3 

22.4 

39.4 

36.0 

25.7 

31.7 

23.7 

10.0 

8.9 

TABLE  XII-B 


BARGRAPH  DATA  FOR  POOLED  PARTICIPANT  TEST  PRECISION 


(All  Participants  Except  Number  7/4  Weeks) 


Gas _  CO 


Probe/Mode 

Concentration 

1/5 

1/4 

2/5 

2/4 

1/3 

1/2 

1/1 

2/3 

2/2 

2/1 

(ppm  CO) 

24.8 

31.9 

40.4 

53.1 

80.0 

143.3 

144.2 

171.7 

476.2  516.9 

+2S/Avg.  % 

30.4 

24.8 

21.6 

17.0 

10.2 

10.0 

9.3 

8.2 

4.0 

3.4 

Gas 

CO, 

Probe/Mode 

Concentration 

1/1 

1/2 

1/3 

1/4 

1/5 

2/1 

2/2 

2/3 

2/4 

2/5 

(%  COo) 

0.58 

0.68 

0.98 

1.48 

1.69  1.74 

1.80 

1.99 

2.54 

2.78 

+2S/Avg7  % 

11.4 

11.8 

10.0 

7.1 

5.9 

5.9 

6.9 

5.6 

5.5 

5.4 

Gas 

THC 

Probe/Mode 

Concentration 

2/5 

2/4 

1/5 

1/4 

1/3 

2/3 

1/2 

1/1 

2/2 

2/1 

(ppmC  THC) 

3.3 

3.7 

4.1 

4.2 

11.7 

24.0 

42.0 

53.8 

192.4 

228.7 

+2S/Avg.  % 

77.3 

73.3 

51.4 

50.3 

17.2 

14.6 

10.3 

9.3 

7.0 

7.8 

Gas 

NO 

y 

Probe/Mode 

Concentration 

1/1 

1/2 

1/3 

2/1 

2/2 

2/3 

1/4 

1/5 

2/4 

2/5 

(ppm  NOJ 
+2S/Avg.  % 

4.9 

6.5 

15.5 

16.0 

18.3 

33.1 

39.7 

54.5 

71.1 

92.8 

31.2 

27.3 

13.7 

8.3 

7.5 

7.2 

6.7 

6.6 

6.0 

6. 1 
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IX.  CONCLUSIONS 


With  few  exceptions,  it  has  been  intended  to  limit  the  analysis  to  the 
major  objective  of  this  cross-correlation  program;  the  conclusions  are 
therefore  similarly  specific. 


1.  An  emission  measurement  cross-correlation  has  been  established 
between  testing  laboratories,  experienced  in  gas  turbine  emission 
measurements,  measuring  the  emissions  of  a  single  test  engine. 


2.  The  variability  between  testing  laboratories  is  a  function  of 
emission  gas  and  level.  The  variability  is  well  in  excess  of  +2% 
for  each  participant  with  the  one  exception  of  CO2  at  high  power. 

3.  Short-term  instrument  precision  errors  have  a  negligible  effect 
on  span  gas  comparisons. 


4.  Individual  participant  test  precision  errors  are  not  a  major 
factor  in  participant-to-participant  test  variability. 


5.  Individual  participant  test  precision  error  as  a  percent  of  level 
is  an  inverse  function  of  gas  concentration. 


6.  Span  gas  bias  errors  from  one  participant  to  another  are  three 
times  those  recommended  as  acceptable  to  the  EPA.  Even  on  clean 
span  gases,  the  EPA  variability  guideline  of  +2%  cannot  be  met. 


7.  Smoke  numbers  obtained  in  this  analysis  were  too  high  as  compared 
to  current  technology,  low  smoke  engines,  and  were  of  insuffi¬ 
cient  quantity  for  meaningful  conclusions.  However,  the  average 
variability  was  much  in  excess  of  the  expected  SAE/ARP  1179 
precision  of  +3  SN. 


X.  RECOMMENDATIONS  AND  FUTURE  EFFORT 


The  results  of  this  test  program  have  indicated  that  the  variability 
of  emissions  measurements  from  participant-to-participant  are  con¬ 
siderably  higher  than  desired  objectives.  The  question  then  arises  as 
to  what  are  the  major  causes  of  the  variations.  The  following  recom¬ 
mendations  are  directed  toward  a  resolution  of  some  of  these 
questions. 


The  regulatory  authorities  should  consider  setting  more  realistic 
variability  guidelines  in  light  of  the  results  of  this  test 
program. 

An  effort  should  be  made  to  establish  more  detailed  criteria  for 
the  operation  of  the  emissions  equipment  to  improve  the  measure¬ 
ment  repeatability.  There  are  many  factors,  such  as  ambient 
temperature  conditions,  sample  line  temperature  and  conditioning, 
ambient  CO?  level,  to  name  a  few,  which  can  influence  the 
analyzer  outputs.  These  should  be  studied  and  better  controlled 
whenever  possible. 


There  are  likely  sufficient  data  available  from  the  test  program 
described  in  this  report  to  permit  a  study  of  the  hysteresis 
effects  resulting  from  engine  power  level  variation.  These  data 
should  be  statistically  analyzed  to  determine  whether  or  not 
hysteresis  causes  a  significant  error  in  the  test  results. 
Hysteresis  will  not  effect  results  of  comparisons  in  this  report, 
since  all  participants  made  their  measurements  at  the  same  time. 
It  may,  however,  effect  measurement  of  absolute  engine  emission 
levels. 


The  smoke  measurement  technique  needs  improvement.  It  may  be 
necessary  to  replace  the  filter  stain  method  with  an  entirely  new 
method,  such  as  an  optical  method,  in  order  to  accomplish  a 
significant  improvement  in  measurement  accuracy. 


Instrument  manufacturers  should  be  encouraged  to  improve  the 
accuracy  of  their  instruments,  particularly  at  low  levels.  The 
total  hydrocarbon  analyzer,  especially,  needs  improvement  in  this 
area. 


To  minimize  the  errors  due  to  calibration  gases,  these  gases 
should  be  periodically  checked  against  reference  standards  which 
are  traceable  to  the  National  Bureau  of  Standards. 


Calibration  gas  "round-robin"  programs  should  be  continued  in  the 
future  to  monitor  the  progress  which  is  being  made  in  the 
improvement  of  calibration  gas  accuracy. 


The  data  from  Phase  II  of  this  program  still  need  to  be  analyzed 
i n  order  to  increase  the  data  base  used  in  arriving  at  the 
program  conclusions. 
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FIGURE  5A 


POOLED  PARTICIPANT  TEST  PRECISION  (2S/AVERAGE) 
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FIGURE  5B 
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APPENDIX  B 


CALIBRATION  GAS  MEASUREMENT  SUMMARY 
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APPENDIX  D 


ENGINE  SMOKE  MEASUREMENTS 


IF  A  DECIMAL  POINT,  .  ,  APPEARS  IN  THE 


TABLE,  IT  INDICATES  THAT  NO  DATA  WERE  TAKEN 


AT  THAT  TEST  CONDITION. 


APPENDIX  D 


ENGINE  SMOKE  MEASUREMENTS 


(SAE  SMOKE  NUMBER) 


Week 

Pt. 

Probe 

Mode 

Port  1 

Port  2 

Port  3 

Port  4 

1 

84 

High 

3 

62.2 

70.0 

74.1 

1 

85 

High 

5 

74.3 

80.0 

73.6 

1 

87 

High 

5 

71.5 

70.0 

80.4 

1 

88 

High 

3 

63.4 

0.0 

79.3 

1 

91 

High 

5 

70.3 

82.0 

84.9 

1 

92 

High 

3 

64.3 

72.0 

81.4 

1 

94 

High 

3 

61.4 

70.0 

71.6 

1 

95 

High 

5 

68.7 

‘82.0 

79.1 

1 

97 

Low 

3 

55.7 

61.0 

71.4 

1 

98 

Low 

5 

69.4 

77.0 

84.2 

1 

100 

Low 

5 

70.7 

77.0 

81.4 

1 

1C1 

Low 

3 

55.2 

60.0 

58.9 

1 

104 

Low 

5 

68.9 

79.0 

75.7 

1 

105 

Low 

3 

56.2 

65.0 

57.7 

1 

107 

Low 

3 

58.5 

56.0 

60.6 

1 

108 

Low 

5 

68.2 

79.0 

63.5 

2 

191 

Low 

1 

29.7 

28.5 

27.0 

28.5 

2 

192 

Low 

2 

35.4 

39.0 

39.0 

37.4 

2 

193 

Low 

3 

55.6 

61.0 

60.0 

59.0 

2 

195 

Low 

3 

56.9 

67.0 

62.0 

74.9 

2 

196 

Low 

2 

30.3 

35.1 

34.0 

39.3 

2 

197 

Low 

1 

22.9 

23.3 

26.3 

3 

280 

Low 

1 

25.2 

23.7 

26.5 

28.4 

3 

281 

Low 

2 

31.8 

32.6 

36.8 

37.2 

3 

282 

Low 

3 

55.7 

60.0 

60.2 

59.6 

3 

284 

Low 

3 

55.7 

61.4 

61.5 

73.5 

3 

285 

Low 

2 

38.5 

36.4 

35.7 

45.5 

3 

286 

Low 

1 

26.1 

29.4 

30.5 

30.1 

3 

288 

Low 

3 

55.2 

64.7 

61.1 

66.8 

3 

289 

Low 

2 

30.3 

34.8 

36.8 

38.2 

3 

290 

Low 

1 

23.7 

28.7 

28.8 

29.4 

3 

291 

Low 

2 

32.6 

38.3 

37.8 

37.9 

1 

i 

Week 

Pt. 

Probe 

J 

Mode 

D-2 

Port  1 

Port  2 

Port  3 

i 

1 

Port  4  2 

$ 

4 

374 

Low 

i 

39.3 

51.4 

39.6 

1 

45.5 

$ 

4 

375 

Low 

2 

42.5 

50.6 

46.9 

45.5  ;•> 

4 

376 

Low 

3 

59.9 

77.4 

65.0 

60.7  '**- 

4 

378 

Low 

3 

58.4 

74.6 

63.3 

63.1 

.  .* 

4 

379 

Low 

2 

32.7 

47.3 

37.8 

39.2  a 

4 

380 

Low 

1 

25.9 

38.4 

29.2 

28.4  M 

1 

4 

382 

Low 

3 

57.4 

75.8 

64.1 

68.2 

4 

383 

Low 

2 

33.4 

48.3 

38.6 

45.2  ,, 

■  .* 

4 

384 

Low 

1 

25.5 

37.5 

28.3 

25.9  W 

? ' 

4 

385 

Low 

2 

34.2 

49.6 

40.0 

37.9  “ 

i 

5 

468 

Low 

1 

24.3 

» 

24.2 

1 

5 

469 

Low 

2 

33.9 

35.9 

5 

470 

Low 

3 

56.3 

59.0  M 

$ 

5 

472 

Low 

3 

59.3 

59.5  M 

5 

473 

Low 

2 

33.4 

36.2  ■ 

!«J*| 

5 

474 

Low 

1 

26.2 

30.0 

5 

476 

Low 

3 

56.9 

63.2  < 

jS 

5 

477 

Low 

2 

39.8 

43.7  & 

Dfl 

5 

478 

Low 

1 

30.7 

34.0 

5 

479 

Low 

2 

36.6 

39"  I 

I 

I 


